REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  the  burden,  to  the  Department  of  Defense,  Executive  Service  Directorate  (0704-0188).  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no 
person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ORGANIZATION. 

1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

05-03-2012  Final 

3.  DATES  COVERED  (From  -  To) 

1  January*  2008  -  30  November  201 1 

4.  TITLE  AND  SUBTITLE 

(YIP-08)  Automated,  Certified  Program- rewriting  for  Software  Security  Enforcement 

5a.  CON 

TRACT NUMBER 

FA9550-08- 1-0044 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

Hamlen,  Kevin  W. 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

The  University  of  Texas  at  Dallas 

800  W.  Campbell  Rd„ 

Richardson,  TX  75080-3021 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Office  of  Scientific  Research 

875  North  Randolph  Street 

Suite  325,  Rm  3112 

Arlington,  VA  22203 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

AFOSR 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

/tor  aCfirVtt-TlrZOia  -D44k 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 


ftpprcwe  -fer  Mobile  ^e\ec\S<2- 

13.  SUPPLEMENTARY  NOTES 

Year  4  of  the  project  finalized,  tested,  and  published  the  Chckov  IRM  verification  system  (see  outcome  2  of  attached  report),  and  extended  the 
Reins  SFI  system  to  Linux-based  architectures  (see  outcome  3  of  attached  report). 

14,  ABSTRACT 

This  project  discovered  and  developed  algorithms  and  tools  for  (1)  automatically  retrofitting  binary'  legacy  software  with  access  controls,  and  (2) 
formally  machine-certifying  that  the  retrofitted  software  satisfies  user-specified  security  policies.  The  research  resulted  in  new  software  security 
systems  for  Java,  Action  Script,  and  x86  native  code  that  provably  secure  legacy  code  without  any  form  of  code-producer  cooperation  (e.g„  source 
code  or  compiler  support) 


15.  SUBJECT  TERMS 

software  security,  validation,  runtime  monitors,  access  controls 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

ABSTRACT 

18.  NUMBER 

OF 

PAGES 

19a.  NAME  OF  RESPONSIBLE  PERSON 

a.  REPORT 

b,  ABSTRACT 

C.  THIS  PAGE 

Kevin  W.  Hamlen 

U 

U 

U 

UU 

10 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

(972)  883-4724 

Standard  Form  298  (Rev  8/98) 

Prescribed  by  ANSI  Std.  Z39  18 
Adobe  t  C 


INSTRUCTIONS  FOR  COMPLETING  SF  298 


1.  REPORT  DATE.  Full  publication  date,  including 
day,  month,  if  available.  Must  cite  at  least  the  year  and 
be  Year  2000  compliant,  e  g.  30-06-1998;  xx-06-1998, 
xx-xx-1998 

2.  REPORT  TYPE.  State  the  type  of  report,  such  as 
final,  technical,  intenm,  memorandum,  master’s  thesis, 
progress,  quarterly,  research,  special,  group  study,  etc. 

3.  DATES  COVERED.  Indicate  the  time  during  which 
the  work  was  performed  and  the  report  was  written, 
e.g.,  Jun  1997  -  Jun  1998;  1-10  Jun  1996;  May -Nov 
1998;  Nov  1998. 

4.  TITLE.  Enter  title  and  subtitle  with  volume  number 
and  part  number,  if  applicable  On  classified 
documents,  enter  the  title  classification  in  parentheses. 

5a.  CONTRACT  NUMBER.  Enter  all  contract  numbers 
as  they  appear  in  the  report,  e.g.  F33615-86-C-5169. 

5b.  GRANT  NUMBER.  Enter  all  grant  numbers  as 
they  appear  in  the  report,  e.g.  AFOSR-82-1234. 

5c.  PROGRAM  ELEMENT  NUMBER.  Enter  all 
program  element  numbers  as  they  appear  in  the  report, 
e.g  61101A. 

5d.  PROJECT  NUMBER.  Enter  all  project  numbers  as 
they  appear  in  the  report,  e  g.  1 F665702D1257;  ILIR. 

5e.  TASK  NUMBER.  Enter  all  task  numbers  as  they 
appear  in  the  report,  e.g  05;  RF0330201;  T41 12. 

5f.  WORK  UNIT  NUMBER.  Enter  all  work  unit 
numbers  as  they  appear  in  the  report,  e.g.  001; 
AFAPL30480105. 

6.  AUTHOR(S).  Enter  name(s)  of  person(s) 
responsible  for  writing  the  report,  performing  the 
research,  or  credited  with  the  content  of  the  report.  The 
form  of  entry  is  the  last  name,  first  name,  middle  initial, 
and  additional  qualifiers  separated  by  commas,  e.g. 
Smith,  Richard,  J,  Jr. 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND 
ADDRESS(ES).  Self-explanatory. 


8.  PERFORMING  ORGANIZATION  REPORT  NUMBER. 

Enter  all  unique  alphanumeric  report  numbers  assigned  by 
the  performing  organization,  e.g.  BRL-1234; 
AFWL-TR-85-401 7-Vol-21  -PT-2. 

9.  SPONSORING/MONITORING  AGENCY  NAME(S) 
AND  ADDRESS(ES).  Enter  the  name  and  address  of  the 
organization(s)  financially  responsible  for  and  monitoring 
the  work. 

10.  SPONSOR/MONITOR’S  ACRONYM(S).  Enter,  if 
available,  e.g.  BRL,  ARDEC,  NADC. 

11.  SPONSOR/MONITOR'S  REPORT  NUMBER(S). 

Enter  report  number  as  assigned  by  the  sponsoring/ 
monitoring  agency,  if  available,  e.g.  BRL-TR-829;  -215 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT.  Use 

agency-mandated  availability  statements  to  indicate  the 
public  availability  or  distribution  limitations  of  the  report.  If 
additional  limitations/  restrictions  or  special  markings  are 
indicated,  follow  agency  authorization  procedures,  e.g. 
RD/FRD,  PROPIN,  ITAR,  etc.  Include  copyright 
information. 

13.  SUPPLEMENTARY  NOTES.  Enter  information  not 
included  elsewhere  such  as:  prepared  in  cooperation 
with;  translation  of;  report  supersedes;  old  edition  number, 
etc. 

14.  ABSTRACT.  A  brief  (approximately  200  words) 
factual  summary  of  the  most  significant  information 

15.  SUBJECT  TERMS.  Key  words  or  phrases  identifying 
major  concepts  in  the  report. 

16.  SECURITY  CLASSIFICATION.  Enter  security 
classification  in  accordance  with  security  classification 
regulations,  e.g.  U,  C,  S,  etc.  If  this  form  contains 
classified  information,  stamp  classification  level  on  the  top 
and  bottom  of  this  page 

17.  LIMITATION  OF  ABSTRACT.  This  block  must  be 
completed  to  assign  a  distribution  limitation  to  the  abstract 
Enter  UU  (Unclassified  Unlimited)  or  SAR  (Same  as 
Report).  An  entry  in  this  block  is  necessary  if  the  abstract 
is  to  be  limited 


Standard  Form  298  Back  (Rev  8/98) 


Final  Report:  YIP-08 

Automated,  Certified  Program-rewriting  for 
Software  Security  Enforcement 

Grant/Contract  Niunber:  FA9550-08- 1-0044 

Kevin  W.  Hamlen 
5  March  2012 


Abstract 

This  project  discovered  and  developed  algorithms  and  tools  for 
(1)  automatically  retrofitting  binary  legacy  software  with  access  con¬ 
trols,  and  (2)  formally  machine-certifying  that  the  retrofitted  software 
satisfies  user-specified  security  policies.  The  research  resulted  in  new 
software  security  systems  for  Java,  ActionScript,  and  x86  native  code 
that  provably  secure  legacy  code  without  any  form  of  code-producer 
cooperation  (e.g.,  source  code  or  compiler  support). 


1  Summary  of  Achievements 

1.1  Research  Outcomes 

Research  supported  by  this  contract  resulted  in  the  development  of  three 
major  software  security  systems  with  associated  discoveries  and  innovations. 
All  publications  and  theses  cited  in  this  report  are  available  for  download 
from  the  following  web  page: 

http :  / /ww .  utdallas .  edu/~hamlen/research .  html 

1.  We  developed  the  Security  Policy  Xml  (SPoX)  tool  suite:  the  first 
fully  declarative,  aspect-oriented  policy  specification  and  in-lined  ref¬ 
erence  monitor  (IRM)  system.  SPoX  includes  tools  for  parsing,  an¬ 
alyzing,  and  visualizing  XML-based  security  policy  specifications  and 
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untrusted  Java  bytecode  binaries.  Design,  implementation,  and  exper¬ 
imental  results  are  detailed  in  the  following  publications  and  theses: 
[2,  9,  10,  11,  12,  13,  14,  18]. 

2.  We  discovered  a  new,  more  powerful  IRM-certification  paradigm  based 
on  model-checking.  This  was  implemented  in  the  Chekov'"  verification 
system,  which  automatically  machine-verifies  the  policy-compliance  of 
IRM-instrumented  Java  and  ActionScript  bytecode  binaries.  Design, 
implementation,  and  experimental  results  are  detailed  in  the  following 
publications  and  thesis:  [1,  3,  4,  8,  9,  15,  16,  17]. 

3.  We  designed  and  implemented  Reins:  a  new,  machine-certified  soft¬ 
ware  fault  isolation  (SFI)  system  for  native  x86  architectures  that  im¬ 
plements  IRMs  for  Intel-based  Windows  and  Linux  systems  without 
any  code-producer  cooperation,  such  as  compile-side  support,  source 
code,  debug  symbols,  or  online  symbol  stores.  Its  design  and  imple¬ 
mentation  are  detailed  in  the  following  publications:  [5,  19’.  Two  ad¬ 
ditional  publications  are  submitted  and  currently  under  review. 

1.2  Executive  Summary  of  Conclusions 

We  met  all  four  of  the  primary  goals  proposed  for  the  project: 

•  Our  ActionScript  and  x86  native  code  IRM  implementations  success¬ 
fully  incorporated  machine-verifiable  code  optimizations  during  secu¬ 
rity  retrofitting.  This  sufficed  to  offset  much  of  the  enforcement  over¬ 
head.  For  x86  native  code,  we  report  overheads  of  less  than  3% — 
substantially  better  than  any  prior  system  of  equivalent  capability  to 
our  knowledge  [5]. 

•  Our  model-checking  approach  to  IRM  certification  successfully  verified 
dataflow-sensitive  optimizations  [4]. 

•  SPoX  facilitated  formal  policy  analyses,  such  as  policy  inconsistency 
detection  and  elimination,  that  are  provably  undecidable  with  tradi¬ 
tional,  non-declarative  aspect-oriented  specification  approaches  [12], 

•  We  successfully  extended  all  of  the  above  technologies  to  untyped,  x86 
native  code  software  for  real-world  operating  systems  (Windows  and 
Linux)  [5]. 
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We  conclude  that  certified,  in-lined  reference  monitoring  is  a  highly  feasible, 
flexible,  and  efficient  approach  to  enforcing  software  security  policies  over 
binary  legacy  software.  Additional  applications  of  the  technology  are  being 
explored  in  several  subsequent  projects,  detailed  in  the  next  section. 

1.3  Contribution  to  Other  Awards  and  Contracts 

The  discoveries  above  have  spawned  three  major  ongoing  research  initiatives, 
currently  supported  by  awards  from  the  National  Science  Foundation  (NSF), 
U.S.  Army,  and  Air  Force  Office  of  Scientific  Research  (AFOSR): 

Securing  Web  Advertisements  (NSF,  TC:Medium,  $1.2M,  2011— 
2014).  In  collaboration  with  the  University  of  Illinois  at  Chicago  (UIC), 
we  are  applying  our  ActionScript  certifying  IRM  system  to  develop  security 
systems  for  mobile  web  advertisements.  Malicious  web  ads  ( malvertisements ) 
are  a  major  ongoing  concern  for  end  users,  publishers,  ad  distribution  net¬ 
works,  and  advertisers.  Our  ongoing  work  leverages  the  IRM  technologies 
developed  and  reported  here  to  provide  provably  sound  and  transparent  pro¬ 
tections  for  web  ad  domains. 

Language-based  Security  for  Polymorphic  Malware  Defense  (NSF 
CAREER,  TC,  S500K,  2011—2016).  Our  successful  extension  of  ma¬ 
chine-certified  SFI/IRM  technologies  to  x86  native  code  architectures  (see 
achievement  3  of  §1.1)  is  a  significant  milestone  toward  extending  power¬ 
ful  language-based  security  technologies  to  COTS  native  code  architectures. 
Last  year  the  PI  received  an  NSF  CAREER  award  for  ongoing  research  that 
develops  language-based  protections  for  binary  software  that  is  potentially 
self-modifying,  untyped,  memory-unsafe,  and  obfuscated  to  resist  disassem¬ 
bly. 

Reactively  Adaptive  Malware  (AFOSR,  FA9550-10- 1-0088,  S450K, 
2011-2014)  (U.S.  Army,  S350K,  2011—2012).  The  binary  analysis  and 
transformation  discoveries  reported  here  are  also  being  applied  for  active 
defense.  Our  ongoing  reactively  adaptive  malware  project  develops  mobile 
code  that  detects,  adapts,  and  avoids  antiviral  defenses  fully  automatically  in 
the  wild.  Such  technologies  are  important  for  anticipating  and  understanding 
next-generation  malware,  and  for  counter-attacking  cyber-attackers. 
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2  Educational  Outcomes 

2.1  Student  Support 

Funding  from  this  award  partially  supported  5  graduate  students: 

•  4  Ph.D.  students:  Micah  Jones  (graduated  December  2011  [9],  now 
employed  by  L-3  Communications),  Meera  Sridhar,  Vishwath  Mohan, 
and  Richard  Wartell  (expected  graduations  within  the  next  1.5  years); 
and 

•  1  Masters  student:  Aditi  Patwardhan  (graduated  June  2010  [14]). 

Micah’s  thesis  [9]  developed  the  SPoX  system  (see  outcome  1  of  §1.1)  and 
its  support  for  the  ChekoV'  verifier  (see  outcome  2  of  §1.1).  Aditi’s  the¬ 
sis  [14]  developed  a  visualization  system  for  SPoX  and  Java  bytecode  [13]. 
Meera’s  ongoing  thesis  work  develped  Cheko'/'  and  is  extending  the  technol¬ 
ogy  to  transparency  verification  of  web  ad  IRMs  (see  §1.3).  Vishwath’s  and 
Richard’s  ongoing  theses  developed  the  Reins  system  (see  outcome  3  of  §1.1) 
and  are  continuing  with  its  application  to  polymorphic  malware  defense  and 
reactively  adaptive  malware  (see  §1.3). 

2.2  Course  Development 

Research  conducted  under  this  contract  contributed  to  the  development  of 
substantial  educational  material  that  augmented  3  different  courses  at  UTD: 

•  CS6V81/7301:  Language-based  Security  (Spring  ’08,  Spring  ’ll)  [aver¬ 
age  student  evaluation:  4.84  /  5  =  Excellent]; 

•  CS6371:  Advanced  Programming  Languages  (Fall  ’08,  Spring  ’09,  Fall 
’09,  Spring  ’10,  Spring  ’ll)  [average  student  evaluation:  4.21  /  5  = 
Very  Good]; 

•  CS4384:  Automata  Theory  (Fall  TO,  Fall  ’ll)  [average  student  evalu¬ 
ation:  4.41  /  5  =  Very  Good] 

CS6V81/7301:  Language-based  Security  is  a  graduate-level  elective  that 
trained  students  in  advanced  software  security  technologies  such  as  IRMs, 
SFI,  information  flow  controls,  malware  analysis,  and  binary  obfuscation. 
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Students  received  direct,  hands-on  experience  with  discoveries  and  tools  re¬ 
sulting  from  this  contract. 

CS6371:  Advanced  Programming  Languages  is  a  grad-level  core  course 
that  teaches  language  and  compiler  design.  As  a  result  of  this  contract,  the 
course  was  significant  augmented  with  examples  and  content  motivated  by 
secure  software  development  and  validation.  Students  learned  type-theoretic 
and  axiomatic  semantical  approaches  to  software  security  analysis. 

CS4384:  Automata  Theory  is  an  undergraduate  core  course  that  teaches 
formal  languages  and  introductory  computational  complexity.  The  course 
was  augmented  with  significant  security  content  including  automata-based 
approaches  to  security  policy  specification  and  analysis. 

Federal  CyberSecurity  Scholarship  For  Service  (NSF,  $1.7M,  2010— 
2014).  The  educational  developments  above  contributed  to  the  establish¬ 
ment  and  enhancement  of  a  new,  NSF-supported  Scholarship  For  Service 
(SFS)  program  at  UTD  in  2010,  which  recruits  and  trains  undergraduates 
and  graduates  for  federal  cyber-security  employment.  The  courses  above 
have  been  instrumental  for  recruiting  students  into  the  program. 
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